2026-10-30 20:04:42,120 [ 518310 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:42, check_args_and_update_paths) 2026-10-30 20:04:42,123 [ 518310 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:86, check_args_and_update_paths) 2026-10-30 20:04:42,123 [ 518310 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:97, check_args_and_update_paths) 2026-10-30 20:04:42,123 [ 518310 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:99, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_shb54x --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS=" -rfEps --run-id=2 --color=no --durations=0 'test_storage_s3_queue/test.py::test_direct_select_file[unordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]' test_storage_s3_queue/test.py::test_drop_table test_storage_s3_queue/test.py::test_exception_during_insert 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered]' test_storage_s3_queue/test.py::test_max_set_age test_storage_s3_queue/test.py::test_max_set_size test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]' 'test_storage_s3_queue/test.py::test_processed_file_setting[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting[5]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5]' 'test_storage_s3_queue/test.py::test_processing_threads[ordered]' 'test_storage_s3_queue/test.py::test_processing_threads[unordered]' test_storage_s3_queue/test.py::test_s3_client_reused test_storage_s3_queue/test.py::test_settings_check 'test_storage_s3_queue/test.py::test_shards[ordered-1]' 'test_storage_s3_queue/test.py::test_shards[ordered-8]' 'test_storage_s3_queue/test.py::test_shards[unordered-1]' 'test_storage_s3_queue/test.py::test_shards[unordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-8]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]' test_storage_s3_queue/test.py::test_upgrade test_storage_s3_queue/test.py::test_upgrade_2 -vvv" altinityinfra/integration-tests-runner:cf95077f4609 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: random-0.2, reportlog-0.4.0, xdist-3.5.0, repeat-0.9.4, order-1.0.1, timeout-2.2.0 timeout: 900.0s timeout method: signal timeout func_only: False collecting ... collected 37 items test_storage_s3_queue/test.py::test_direct_select_file[unordered] ERROR [ 2%] test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] ERROR [ 5%] test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] ERROR [ 8%] test_storage_s3_queue/test.py::test_drop_table ERROR [ 10%] test_storage_s3_queue/test.py::test_exception_during_insert ERROR [ 13%] test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered] ERROR [ 16%] test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered] ERROR [ 18%] test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered] ERROR [ 21%] test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered] ERROR [ 24%] test_storage_s3_queue/test.py::test_max_set_age ERROR [ 27%] test_storage_s3_queue/test.py::test_max_set_size ERROR [ 29%] test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch ERROR [ 32%] test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] ERROR [ 35%] test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] ERROR [ 37%] test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] ERROR [ 40%] test_storage_s3_queue/test.py::test_processed_file_setting[1] ERROR [ 43%] test_storage_s3_queue/test.py::test_processed_file_setting[5] ERROR [ 45%] test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1] ERROR [ 48%] test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5] ERROR [ 51%] test_storage_s3_queue/test.py::test_processing_threads[ordered] ERROR [ 54%] test_storage_s3_queue/test.py::test_processing_threads[unordered] ERROR [ 56%] test_storage_s3_queue/test.py::test_s3_client_reused ERROR [ 59%] test_storage_s3_queue/test.py::test_settings_check ERROR [ 62%] test_storage_s3_queue/test.py::test_shards[ordered-1] ERROR [ 64%] test_storage_s3_queue/test.py::test_shards[ordered-8] ERROR [ 67%] test_storage_s3_queue/test.py::test_shards[unordered-1] ERROR [ 70%] test_storage_s3_queue/test.py::test_shards[unordered-8] ERROR [ 72%] test_storage_s3_queue/test.py::test_shards_distributed[ordered-1] ERROR [ 75%] test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] ERROR [ 78%] test_storage_s3_queue/test.py::test_shards_distributed[unordered-1] ERROR [ 81%] test_storage_s3_queue/test.py::test_shards_distributed[unordered-8] ERROR [ 83%] test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] ERROR [ 86%] test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] ERROR [ 89%] test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] ERROR [ 91%] test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] ERROR [ 94%] test_storage_s3_queue/test.py::test_upgrade ERROR [ 97%] test_storage_s3_queue/test.py::test_upgrade_2 ERROR [100%] ==================================== ERRORS ==================================== _____________ ERROR at setup of test_direct_select_file[unordered] _____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ---------------------------- Captured stderr setup ----------------------------- Traceback (most recent call last): File "/ClickHouse/tests/integration/helpers/cluster.py", line 3119, in start self.wait_azurite_to_start() File "/ClickHouse/tests/integration/helpers/cluster.py", line 2706, in wait_azurite_to_start raise Exception("Can't wait Azurite to start") Exception: Can't wait Azurite to start ------------------------------ Captured log setup ------------------------------ 2026-10-30 19:04:47 [ 396 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:113, run_and_check) 2026-10-30 19:04:47 [ 396 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-10-30 19:04:47 [ 396 ] DEBUG : No running containers (conftest.py:95, cleanup_environment) 2026-10-30 19:04:47 [ 396 ] DEBUG : Pruning Docker networks (conftest.py:97, cleanup_environment) 2026-10-30 19:04:47 [ 396 ] DEBUG : Command:['docker network prune --force'] (cluster.py:113, run_and_check) 2026-10-30 19:04:47 [ 396 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:113, run_and_check) 2026-10-30 19:04:47 [ 396 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:121, run_and_check) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_KERBEROS_KDC_TAG 9391ecdee8d7 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV CLICKHOUSE_TESTS_SERVER_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV MSAN_OPTIONS abort_on_error=1 poison_in_dtor=1 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV JAVA_TOOL_OPTIONS -Djdk.attach.allowAttachSelf=true (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV TSAN_OPTIONS halt_on_error=1 abort_on_error=1 history_size=7 memory_limit_mb=46080 second_deadlock_stack=1 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV HOSTNAME c1b62764f25e (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV SHLVL 0 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV HOME /root (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV OLDPWD / (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_HELPER_TAG 2cffe1eae894 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV PYTHONUNBUFFERED 1 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_PYTHON_BOTTLE_TAG caad4729259e (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV UBSAN_OPTIONS print_stacktrace=1 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV PYTEST_ADDOPTS -rfEps --run-id=2 --color=no --durations=0 'test_storage_s3_queue/test.py::test_direct_select_file[unordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]' test_storage_s3_queue/test.py::test_drop_table test_storage_s3_queue/test.py::test_exception_during_insert 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered]' test_storage_s3_queue/test.py::test_max_set_age test_storage_s3_queue/test.py::test_max_set_size test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]' 'test_storage_s3_queue/test.py::test_processed_file_setting[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting[5]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5]' 'test_storage_s3_queue/test.py::test_processing_threads[ordered]' 'test_storage_s3_queue/test.py::test_processing_threads[unordered]' test_storage_s3_queue/test.py::test_s3_client_reused test_storage_s3_queue/test.py::test_settings_check 'test_storage_s3_queue/test.py::test_shards[ordered-1]' 'test_storage_s3_queue/test.py::test_shards[ordered-8]' 'test_storage_s3_queue/test.py::test_shards[unordered-1]' 'test_storage_s3_queue/test.py::test_shards[unordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-8]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]' test_storage_s3_queue/test.py::test_upgrade test_storage_s3_queue/test.py::test_upgrade_2 -vvv (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV CLICKHOUSE_LIBRARY_BRIDGE_BINARY_PATH /clickhouse-library-bridge (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV COMPOSE_HTTP_TIMEOUT 600 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_MYSQL_PHP_CLIENT_TAG 88be89c1e3b6 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_DOTNET_CLIENT_TAG 11de0b29a15d (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV CLICKHOUSE_TESTS_CLIENT_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_MYSQL_JS_CLIENT_TAG 41ba7c2ec2a1 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV PATH /spark-3.3.2-bin-hadoop3/bin:/opt/gdb/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_KERBERIZED_HADOOP_TAG latest (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_CHANNEL stable (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_CLIENT_TIMEOUT 300 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_POSTGRESQL_JAVA_CLIENT_TAG a4eff5c7f4d6 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_NGINX_DAV_TAG b55ac9cd7519 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_MYSQL_GOLANG_CLIENT_TAG 9bec2a638e6e (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV PWD /ClickHouse/tests/integration (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_MYSQL_JAVA_CLIENT_TAG 766bff31cfe4 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV CLICKHOUSE_ODBC_BRIDGE_BINARY_PATH /clickhouse-odbc-bridge (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV CLICKHOUSE_TESTS_BASE_CONFIG_DIR /clickhouse-config (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV TZ Etc/UTC (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV JAVA_PATH /usr/lib/jvm/java-11-openjdk-arm64/bin/java (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV DOCKER_BASE_TAG b74d853ebe76 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV SPARK_HOME /spark-3.3.2-bin-hadoop3 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV LC_CTYPE C.UTF-8 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV INTEGRATION_TESTS_RUN_ID 2 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV WORKER_FREE_PORTS 30000 30001 30002 30003 30004 30005 30006 30007 30008 30009 30010 30011 30012 30013 30014 30015 30016 30017 30018 30019 30020 30021 30022 30023 30024 30025 30026 30027 30028 30029 30030 30031 30032 30033 30034 30035 30036 30037 30038 30039 30040 30041 30042 30043 30044 30045 30046 30047 30048 30049 (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : ENV PYTEST_CURRENT_TEST test_storage_s3_queue/test.py::test_direct_select_file[unordered] (setup) (cluster.py:446, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : CLUSTER INIT base_config_dir:/clickhouse-config (cluster.py:766, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : Removed :/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2 (cluster.py:769, __init__) 2026-10-30 19:04:47 [ 396 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup Keeper (cluster.py:1059, setup_keeper_cmd) 2026-10-30 19:04:47 [ 396 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-10-30 19:04:47 [ 396 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-10-30 19:04:47 [ 396 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance2 tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-10-30 19:04:47 [ 396 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-10-30 19:04:47 [ 396 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:old_instance tag:23.12 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-10-30 19:04:47 [ 396 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-10-30 19:04:47 [ 396 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance_too_many_parts tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-10-30 19:04:47 [ 396 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-10-30 19:04:47 [ 396 ] DEBUG : Cluster name: project_name:rootteststorages3queue. Added instance name:instance_24.5 tag:24.5 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-10-30 19:04:47 [ 396 ] INFO : Starting cluster... (test.py:143, started_cluster) 2026-10-30 19:04:47 [ 396 ] INFO : Running tests in /ClickHouse/tests/integration/test_storage_s3_queue/test.py (cluster.py:2790, start) 2026-10-30 19:04:47 [ 396 ] DEBUG : Cluster start called. is_up=False (cluster.py:2797, start) 2026-10-30 19:04:47 [ 396 ] DEBUG : Docker networks for project rootteststorages3queue are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-10-30 19:04:47 [ 396 ] DEBUG : Docker containers for project rootteststorages3queue are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-10-30 19:04:47 [ 396 ] DEBUG : Docker volumes for project rootteststorages3queue are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-10-30 19:04:47 [ 396 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-10-30 19:04:47 [ 396 ] DEBUG : Docker networks for project rootteststorages3queue are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-10-30 19:04:47 [ 396 ] DEBUG : Docker containers for project rootteststorages3queue are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-10-30 19:04:47 [ 396 ] DEBUG : Docker volumes for project rootteststorages3queue are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-10-30 19:04:47 [ 396 ] DEBUG : Command:docker container list --all --filter name='^/rootteststorages3queue_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-10-30 19:04:47 [ 396 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-10-30 19:04:47 [ 396 ] DEBUG : No running containers for project: rootteststorages3queue (cluster.py:904, cleanup) 2026-10-30 19:04:47 [ 396 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-10-30 19:04:47 [ 396 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-10-30 19:04:47 [ 396 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-10-30 19:04:47 [ 396 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup directory for instance: instance (cluster.py:2810, start) 2026-10-30 19:04:47 [ 396 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/zookeeper.xml', '/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/configs/config.d (cluster.py:4636, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/database (cluster.py:4653, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/logs (cluster.py:4664, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup directory for instance: instance2 (cluster.py:2810, start) 2026-10-30 19:04:47 [ 396 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/configs/config.d (cluster.py:4636, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/database (cluster.py:4653, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/logs (cluster.py:4664, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup directory for instance: old_instance (cluster.py:2810, start) 2026-10-30 19:04:47 [ 396 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Copy custom test config files [] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/configs/config.d (cluster.py:4636, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/database (cluster.py:4653, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/logs (cluster.py:4664, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup directory for instance: instance_too_many_parts (cluster.py:2810, start) 2026-10-30 19:04:47 [ 396 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml', '/ClickHouse/tests/integration/test_storage_s3_queue/configs/merge_tree.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/configs/config.d (cluster.py:4636, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/database (cluster.py:4653, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/logs (cluster.py:4664, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup directory for instance: instance_24.5 (cluster.py:2810, start) 2026-10-30 19:04:47 [ 396 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Copy custom test config files [] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/configs/config.d (cluster.py:4636, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/database (cluster.py:4653, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/logs (cluster.py:4664, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-10-30 19:04:47 [ 396 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:b74d853ebe76', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/coordination', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/certs/public.crt', 'AZURITE_PORT': '30000', 'AZURITE_STORAGE_ACCOUNT_URL': 'http://azurite1:30000/devstoreaccount1', 'AZURITE_CONNECTION_STRING': 'DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:30000/devstoreaccount1;'} stored in /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env (cluster.py:86, _create_env_file) 2026-10-30 19:04:47 [ 396 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-10-30 19:04:47 [ 396 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-10-30 19:04:47 [ 396 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-10-30 19:04:47 [ 396 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-10-30 19:04:48 [ 396 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 825 (connectionpool.py:547, _make_request) 2026-10-30 19:04:48 [ 396 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling azurite1 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo3 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo1 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo2 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance2 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling old_instance ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance_24.5 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance_too_many_parts ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance ... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance_too_many_parts ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling azurite1 ... pulling from altinityinfra/azurite (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance_too_many_parts ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance_too_many_parts ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling azurite1 ... digest: sha256:dae2a5f96553962901... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling azurite1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance_too_many_parts ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling azurite1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance2 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling minio1 ... pulling from clickhouse/minio-minio (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling old_instance ... pulling from clickhouse/clickhous... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling minio1 ... digest: sha256:9eea83a4e1425067e7... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling minio1 ... status: image is up to date for c... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling minio1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling old_instance ... digest: sha256:5d0802b64c09f6d2c6... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling old_instance ... status: image is up to date for c... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling old_instance ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo2 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance_24.5 ... pulling from clickhouse/clickhous... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance_24.5 ... digest: sha256:f1155aa4adec32964c... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance_24.5 ... status: image is up to date for c... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling instance_24.5 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Stderr:Pulling zoo3 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:51 [ 396 ] DEBUG : Setup ZooKeeper (cluster.py:2853, start) 2026-10-30 19:04:51 [ 396 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/coordination', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/coordination'] (cluster.py:2854, start) 2026-10-30 19:04:51 [ 396 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'arm64', 'BuildTime': '2023-05-05T21:18:33.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-198-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=arm64, KernelVersion=5.15.0-198-generic, BuildTime=2023-05-05T21:18:33.000000000+00:00 (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'aarch64', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Containers': 0, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.network.ensure: Creating network "rootteststorages3queue_default" with the default driver (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network <- (name='rootteststorages3queue_default', driver=None, options=None, ipam=None, internal=False, enable_ipv6=False, labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.network': 'default', 'com.docker.compose.version': '1.29.2'}, attachable=True, check_duplicate=True) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network -> {'Id': '0ad0c62ab4808bf54c58958a85ff312bcaaacd56c4f0cc1c9187b6b8b48d206f', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Warning': ''} (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, , } (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo2_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='zoo2', number=1)} (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='zoo2', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo3_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='zoo3', number=1)} (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='zoo3', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='zoo1', number=1)} (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='zoo1', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: cafae9e2b19320913689528f24add9aff8c81cce6ce0496d5b479e8ab1908680 (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/log', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Source': '/clickhouse', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Target': '/usr/bin/clickhouse', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config2.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_zoo2_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'zoo2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'cafae9e2b19320913689528f24add9aff8c81cce6ce0496d5b479e8ab1908680'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper2/log', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['zoo2'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 7517fb96967f1831c2a564ba9ccd8f6dd3b83f6adf6eac95f830347cfe819945 (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/log', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Source': '/clickhouse', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Target': '/usr/bin/clickhouse', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config3.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_zoo3_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'zoo3', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '7517fb96967f1831c2a564ba9ccd8f6dd3b83f6adf6eac95f830347cfe819945'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper3/log', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['zoo3'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: b42afcc5df401b762e0badcf48a294d3ebc56f8caff515d47ed3e903547ecd47 (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/config', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Source': '/clickhouse', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Target': '/usr/bin/clickhouse', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config1.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_zoo1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'zoo1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'b42afcc5df401b762e0badcf48a294d3ebc56f8caff515d47ed3e903547ecd47'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/keeper1/config', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['zoo1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'c1029d9a25eb8b3cca3763cf22c5b32a214ae7691a8c9141a3e582cae55f2b5a', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('c1029d9a25eb8b3cca3763cf22c5b32a214ae7691a8c9141a3e582cae55f2b5a') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'f7c030d02b4cb9d2ad46095f146598fb82f70edccffa0e0ffc4db8f51b8ad61a', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('f7c030d02b4cb9d2ad46095f146598fb82f70edccffa0e0ffc4db8f51b8ad61a') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('c1029d9a25eb8b3cca3763cf22c5b32a214ae7691a8c9141a3e582cae55f2b5a', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87', 'rootteststorages3queue_default', aliases=['zoo1', '884c4bc9ef42'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('f7c030d02b4cb9d2ad46095f146598fb82f70edccffa0e0ffc4db8f51b8ad61a', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('f7c030d02b4cb9d2ad46095f146598fb82f70edccffa0e0ffc4db8f51b8ad61a', 'rootteststorages3queue_default', aliases=['zoo3', 'f7c030d02b4c'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('c1029d9a25eb8b3cca3763cf22c5b32a214ae7691a8c9141a3e582cae55f2b5a', 'rootteststorages3queue_default', aliases=['c1029d9a25eb', 'zoo2'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('f7c030d02b4cb9d2ad46095f146598fb82f70edccffa0e0ffc4db8f51b8ad61a') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('c1029d9a25eb8b3cca3763cf22c5b32a214ae7691a8c9141a3e582cae55f2b5a') (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='zoo3', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='zoo1', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='zoo2', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:04:54 [ 396 ] DEBUG : Wait ZooKeeper to start (cluster.py:2506, wait_zookeeper_to_start) 2026-10-30 19:04:54 [ 396 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:2137, get_instance_ip) 2026-10-30 19:04:54 [ 396 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_zoo1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-10-30 19:04:54 [ 396 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.1.2, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-10-30 19:04:54 [ 396 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 19:04:54 [ 396 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 19:04:54 [ 396 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 19:04:54 [ 396 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 19:04:55 [ 396 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 19:04:55 [ 396 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 19:04:55 [ 396 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 19:04:55 [ 396 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 19:04:56 [ 396 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 19:04:56 [ 396 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 19:04:57 [ 396 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 19:04:57 [ 396 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 19:04:59 [ 396 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 19:04:59 [ 396 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-10-30 19:04:59 [ 396 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-10-30 19:04:59 [ 396 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-10-30 19:04:59 [ 396 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-10-30 19:04:59 [ 396 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-10-30 19:04:59 [ 396 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-10-30 19:04:59 [ 396 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-10-30 19:04:59 [ 396 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-10-30 19:04:59 [ 396 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-10-30 19:04:59 [ 396 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-10-30 19:04:59 [ 396 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:2137, get_instance_ip) 2026-10-30 19:04:59 [ 396 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_zoo2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-10-30 19:04:59 [ 396 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.1.4, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-10-30 19:04:59 [ 396 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 19:04:59 [ 396 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-10-30 19:04:59 [ 396 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-10-30 19:04:59 [ 396 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-10-30 19:04:59 [ 396 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-10-30 19:04:59 [ 396 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-10-30 19:04:59 [ 396 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-10-30 19:04:59 [ 396 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-10-30 19:04:59 [ 396 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-10-30 19:04:59 [ 396 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-10-30 19:04:59 [ 396 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-10-30 19:04:59 [ 396 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:2137, get_instance_ip) 2026-10-30 19:04:59 [ 396 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_zoo3_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-10-30 19:04:59 [ 396 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.1.3, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-10-30 19:04:59 [ 396 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 19:04:59 [ 396 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-10-30 19:04:59 [ 396 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-10-30 19:04:59 [ 396 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-10-30 19:04:59 [ 396 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-10-30 19:04:59 [ 396 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-10-30 19:04:59 [ 396 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-10-30 19:04:59 [ 396 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-10-30 19:04:59 [ 396 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-10-30 19:04:59 [ 396 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-10-30 19:04:59 [ 396 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-10-30 19:04:59 [ 396 ] DEBUG : All instances of ZooKeeper Secure started (cluster.py:2521, wait_zookeeper_nodes_to_start) 2026-10-30 19:04:59 [ 396 ] INFO : Trying to create Minio instance by command docker-compose --env-file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env --project-name rootteststorages3queue --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --verbose up -d (cluster.py:3091, start) 2026-10-30 19:04:59 [ 396 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'arm64', 'BuildTime': '2023-05-05T21:18:33.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-198-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=arm64, KernelVersion=5.15.0-198-generic, BuildTime=2023-05-05T21:18:33.000000000+00:00 (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Containers': {'884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87': {'EndpointID': '82554f9d7d6e514e56f949e4f5c429691a8fd5d769dfd9b211f8d1ef682d9b63', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.2/24', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:02', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_zoo1_1'}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'c1029d9a25eb8b3cca3763cf22c5b32a214ae7691a8c9141a3e582cae55f2b5a': {'EndpointID': '02cfec5ce87af0a0d6224c08d3fbcd1c24c3ed8cff5f787cbcd67109ae1d3a9e', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.4/24', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'aarch64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Containers': 3, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Containers': {'884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87': {'EndpointID': '82554f9d7d6e514e56f949e4f5c429691a8fd5d769dfd9b211f8d1ef682d9b63', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.2/24', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:02', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_zoo1_1'}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'c1029d9a25eb8b3cca3763cf22c5b32a214ae7691a8c9141a3e582cae55f2b5a': {'EndpointID': '02cfec5ce87af0a0d6224c08d3fbcd1c24c3ed8cff5f787cbcd67109ae1d3a9e', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.4/24', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.volume.initialize: Creating volume "rootteststorages3queue_data1-1" with default driver (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_volume <- ('rootteststorages3queue_data1-1', None, None, labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.volume': 'data1-1', 'com.docker.compose.version': '1.29.2'}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_volume -> {'CreatedAt': '2026-10-30T19:05:00Z', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 3 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 3 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('f7c030d02b4cb9d2ad46095f146598fb82f70edccffa0e0ffc4db8f51b8ad61a') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('c1029d9a25eb8b3cca3763cf22c5b32a214ae7691a8c9141a3e582cae55f2b5a') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.project.find_orphan_containers: Found orphan containers (rootteststorages3queue_zoo1_1, rootteststorages3queue_zoo3_1, rootteststorages3queue_zoo2_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('clickhouse/minio-minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.project._get_convergence_plans: minio1 has upstream changes (proxy1, proxy2) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.project._get_convergence_plans: resolver has upstream changes (proxy1, proxy2) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, , , } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='proxy1', number=1)} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='proxy1', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy2_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='proxy2', number=1)} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='proxy2', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: be276a2c71c70082fb5917109b7a6664c7e08cd24202d86138aa6bd4db165a4f (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/s3-proxy', volumes={}, name='rootteststorages3queue_proxy1_1', detach=True, ports=['8080', '80', '443'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'proxy1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'be276a2c71c70082fb5917109b7a6664c7e08cd24202d86138aa6bd4db165a4f'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['proxy1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: be276a2c71c70082fb5917109b7a6664c7e08cd24202d86138aa6bd4db165a4f (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/s3-proxy', volumes={}, name='rootteststorages3queue_proxy2_1', detach=True, ports=['8080', '80', '443'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'proxy2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'be276a2c71c70082fb5917109b7a6664c7e08cd24202d86138aa6bd4db165a4f'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['proxy2'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '98b685380a8f96dd0b8230e7d11484f81d211ee820056bc0f2902a4521ceb073', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('98b685380a8f96dd0b8230e7d11484f81d211ee820056bc0f2902a4521ceb073') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '9ace7928eb57000a5ecae804bce15f456beef419ff9e779e1de0378880ab6059', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('9ace7928eb57000a5ecae804bce15f456beef419ff9e779e1de0378880ab6059') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('98b685380a8f96dd0b8230e7d11484f81d211ee820056bc0f2902a4521ceb073', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('9ace7928eb57000a5ecae804bce15f456beef419ff9e779e1de0378880ab6059', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('9ace7928eb57000a5ecae804bce15f456beef419ff9e779e1de0378880ab6059', 'rootteststorages3queue_default', aliases=['proxy2', '9ace7928eb57'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('98b685380a8f96dd0b8230e7d11484f81d211ee820056bc0f2902a4521ceb073', 'rootteststorages3queue_default', aliases=['proxy1', '98b685380a8f'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('9ace7928eb57000a5ecae804bce15f456beef419ff9e779e1de0378880ab6059') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('98b685380a8f96dd0b8230e7d11484f81d211ee820056bc0f2902a4521ceb073') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='proxy2', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='proxy1', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_resolver_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='resolver', number=1)} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='resolver', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_minio1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='minio1', number=1)} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='minio1', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('clickhouse/minio-minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('clickhouse/minio-minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 9c8e64e1c17dadce3ec31247717fd82b2ddc331a734d26def0549760b760948c (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/python-bottle:caad4729259e', tty=True, volumes={}, name='rootteststorages3queue_resolver_1', detach=True, ports=['8080'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'resolver', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '9c8e64e1c17dadce3ec31247717fd82b2ddc331a734d26def0549760b760948c'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['resolver'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 6f2cffe9372e046162310bdd34e2bf7a0476aa30e4d9c34e07491940f2ae307f (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=['/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/certs:/certs:rw', 'rootteststorages3queue_data1-1:/data1:rw'], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': ['/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/certs:/certs:rw', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'rootteststorages3queue_data1-1:/data1:rw'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (command='server --console-address 127.0.0.1:19001 --address :9001 --certs-dir /certs /data1-1', environment=['MINIO_ACCESS_KEY=minio', 'MINIO_SECRET_KEY=minio123', 'MINIO_PROMETHEUS_AUTH_TYPE=public'], image='clickhouse/minio-minio:RELEASE.2024-07-31T05-46-26Z', volumes={'/certs': {}, '/data1': {}}, name='rootteststorages3queue_minio1_1', detach=True, ports=['9001'], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'minio1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '6f2cffe9372e046162310bdd34e2bf7a0476aa30e4d9c34e07491940f2ae307f'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': ['/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/minio/certs:/certs:rw', 'rootteststorages3queue_data1-1:/data1:rw'], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['minio1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '26c8983250a2dabc7fafbf3884c2ece598537f759f45fcfab5a4a5bf39d9faeb', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('26c8983250a2dabc7fafbf3884c2ece598537f759f45fcfab5a4a5bf39d9faeb') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Args': [], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('26c8983250a2dabc7fafbf3884c2ece598537f759f45fcfab5a4a5bf39d9faeb', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'f2938cb41e9df53d0f1660e371998de36b2967cddba8a37c5f668bee90f4982f', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('f2938cb41e9df53d0f1660e371998de36b2967cddba8a37c5f668bee90f4982f') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('26c8983250a2dabc7fafbf3884c2ece598537f759f45fcfab5a4a5bf39d9faeb', 'rootteststorages3queue_default', aliases=['26c8983250a2', 'resolver'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Args': ['server', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--console-address', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '127.0.0.1:19001', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--address', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: ':9001', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '--certs-dir', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '/certs', (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: '/data1-1'], (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('f2938cb41e9df53d0f1660e371998de36b2967cddba8a37c5f668bee90f4982f', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('26c8983250a2dabc7fafbf3884c2ece598537f759f45fcfab5a4a5bf39d9faeb') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('f2938cb41e9df53d0f1660e371998de36b2967cddba8a37c5f668bee90f4982f', 'rootteststorages3queue_default', aliases=['f2938cb41e9d', 'minio1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('f2938cb41e9df53d0f1660e371998de36b2967cddba8a37c5f668bee90f4982f') (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='resolver', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_resolver_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='minio1', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_minio1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:03 [ 396 ] INFO : Trying to connect to Minio... (cluster.py:3097, start) 2026-10-30 19:05:03 [ 396 ] DEBUG : get_instance_ip instance_name=minio1 (cluster.py:2137, get_instance_ip) 2026-10-30 19:05:03 [ 396 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_minio1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-10-30 19:05:03 [ 396 ] DEBUG : get_instance_ip instance_name=proxy1 (cluster.py:2137, get_instance_ip) 2026-10-30 19:05:03 [ 396 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_proxy1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-10-30 19:05:03 [ 396 ] DEBUG : Starting new HTTP connection (1): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-10-30 19:05:03 [ 396 ] DEBUG : Incremented Retry for (url='/'): Retry(total=2, connect=None, read=None, redirect=None, status=None) (retry.py:517, increment) 2026-10-30 19:05:03 [ 396 ] WARNING : Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / (connectionpool.py:872, urlopen) 2026-10-30 19:05:03 [ 396 ] DEBUG : Starting new HTTP connection (2): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-10-30 19:05:03 [ 396 ] DEBUG : Incremented Retry for (url='/'): Retry(total=1, connect=None, read=None, redirect=None, status=None) (retry.py:517, increment) 2026-10-30 19:05:03 [ 396 ] WARNING : Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / (connectionpool.py:872, urlopen) 2026-10-30 19:05:03 [ 396 ] DEBUG : Starting new HTTP connection (3): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-10-30 19:05:03 [ 396 ] DEBUG : Incremented Retry for (url='/'): Retry(total=0, connect=None, read=None, redirect=None, status=None) (retry.py:517, increment) 2026-10-30 19:05:03 [ 396 ] WARNING : Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / (connectionpool.py:872, urlopen) 2026-10-30 19:05:03 [ 396 ] DEBUG : Starting new HTTP connection (4): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-10-30 19:05:03 [ 396 ] DEBUG : Can't connect to Minio: HTTPConnectionPool(host='172.16.1.8', port=9001): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) (cluster.py:2671, wait_minio_to_start) 2026-10-30 19:05:04 [ 396 ] DEBUG : Starting new HTTP connection (5): 172.16.1.8:9001 (connectionpool.py:245, _new_conn) 2026-10-30 19:05:04 [ 396 ] DEBUG : http://172.16.1.8:9001 "GET / HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-10-30 19:05:04 [ 396 ] DEBUG : Connected to Minio. (cluster.py:2651, wait_minio_to_start) 2026-10-30 19:05:04 [ 396 ] DEBUG : http://172.16.1.8:9001 "GET /root?location= HTTP/1.1" 404 0 (connectionpool.py:547, _make_request) 2026-10-30 19:05:04 [ 396 ] DEBUG : http://172.16.1.8:9001 "PUT /root HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-10-30 19:05:04 [ 396 ] DEBUG : S3 bucket 'root' created (cluster.py:2666, wait_minio_to_start) 2026-10-30 19:05:04 [ 396 ] DEBUG : http://172.16.1.8:9001 "GET /root2?location= HTTP/1.1" 404 0 (connectionpool.py:547, _make_request) 2026-10-30 19:05:05 [ 396 ] DEBUG : http://172.16.1.8:9001 "PUT /root2 HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-10-30 19:05:05 [ 396 ] DEBUG : S3 bucket 'root2' created (cluster.py:2666, wait_minio_to_start) 2026-10-30 19:05:05 [ 396 ] INFO : Trying to create Azurite instance by command docker-compose --env-file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env --project-name rootteststorages3queue --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml --verbose up -d (cluster.py:3102, start) 2026-10-30 19:05:05 [ 396 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'arm64', 'BuildTime': '2023-05-05T21:18:33.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-198-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=arm64, KernelVersion=5.15.0-198-generic, BuildTime=2023-05-05T21:18:33.000000000+00:00 (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Containers': {'26c8983250a2dabc7fafbf3884c2ece598537f759f45fcfab5a4a5bf39d9faeb': {'EndpointID': 'eac295b66f8ff972989de0b59611ee4c10b78593807f66c1e2459609191ce07e', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.7/24', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:07', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_resolver_1'}, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87': {'EndpointID': '82554f9d7d6e514e56f949e4f5c429691a8fd5d769dfd9b211f8d1ef682d9b63', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.2/24', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-10-30T19:05:00Z', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'aarch64', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Containers': 7, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Containers': {'26c8983250a2dabc7fafbf3884c2ece598537f759f45fcfab5a4a5bf39d9faeb': {'EndpointID': 'eac295b66f8ff972989de0b59611ee4c10b78593807f66c1e2459609191ce07e', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.7/24', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:01:07', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_resolver_1'}, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87': {'EndpointID': '82554f9d7d6e514e56f949e4f5c429691a8fd5d769dfd9b211f8d1ef682d9b63', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'IPv4Address': '172.16.1.2/24', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-10-30T19:05:00Z', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_data1-1') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-10-30T19:05:00Z', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_data1-1/_data', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_data1-1', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 7 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 7 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('c1029d9a25eb8b3cca3763cf22c5b32a214ae7691a8c9141a3e582cae55f2b5a') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('f2938cb41e9df53d0f1660e371998de36b2967cddba8a37c5f668bee90f4982f') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Args': ['server', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--console-address', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '127.0.0.1:19001', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--address', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: ':9001', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--certs-dir', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '/certs', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '/data1-1'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('26c8983250a2dabc7fafbf3884c2ece598537f759f45fcfab5a4a5bf39d9faeb') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Args': [], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('f7c030d02b4cb9d2ad46095f146598fb82f70edccffa0e0ffc4db8f51b8ad61a') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('9ace7928eb57000a5ecae804bce15f456beef419ff9e779e1de0378880ab6059') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('98b685380a8f96dd0b8230e7d11484f81d211ee820056bc0f2902a4521ceb073') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('884c4bc9ef428091359cfd0563caa656c73159351fe2d59b6c399d2d8ef56d87') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.project.find_orphan_containers: Found orphan containers (rootteststorages3queue_zoo2_1, rootteststorages3queue_minio1_1, rootteststorages3queue_resolver_1, rootteststorages3queue_zoo3_1, rootteststorages3queue_proxy2_1, rootteststorages3queue_proxy1_1, rootteststorages3queue_zoo1_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {} (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_azurite1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue', service='azurite1', number=1)} (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue', service='azurite1', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 2ece5e07b89b0f4e7b5e8076ad1e1b0717274882ab654de56a4c7613ded50eb6 (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={'30000/tcp': ['30000']}, binds=['rootteststorages3queue_data1-1:/data1:rw'], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': ['rootteststorages3queue_data1-1:/data1:rw'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_default', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'PortBindings': {'30000/tcp': [{'HostIp': '', 'HostPort': '30000'}]}, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (command='azurite-blob --blobHost 0.0.0.0 --blobPort 30000 --debug /azurite_log', image='altinityinfra/azurite:3.35.0', ports=[('30000', 'tcp')], volumes={'/data1': {}}, name='rootteststorages3queue_azurite1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue', 'com.docker.compose.service': 'azurite1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_azurite.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '2ece5e07b89b0f4e7b5e8076ad1e1b0717274882ab654de56a4c7613ded50eb6'}, host_config={'NetworkMode': 'rootteststorages3queue_default', 'VolumesFrom': [], 'Binds': ['rootteststorages3queue_data1-1:/data1:rw'], 'PortBindings': {'30000/tcp': [{'HostIp': '', 'HostPort': '30000'}]}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_default': {'Aliases': ['azurite1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '3e8cfe9bf82ec94dba515c9e2f2f4c896518a702afa5c3c4a4efa5a5989fb7eb', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Warnings': ["The requested image's platform (linux/amd64) does not match the " (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'detected host platform (linux/arm64/v8) and no specific ' (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'platform was requested']} (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('3e8cfe9bf82ec94dba515c9e2f2f4c896518a702afa5c3c4a4efa5a5989fb7eb') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Args': ['azurite-blob', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--blobHost', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '0.0.0.0', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--blobPort', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '30000', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '--debug', (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: '/azurite_log'], (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('3e8cfe9bf82ec94dba515c9e2f2f4c896518a702afa5c3c4a4efa5a5989fb7eb', 'rootteststorages3queue_default') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('3e8cfe9bf82ec94dba515c9e2f2f4c896518a702afa5c3c4a4efa5a5989fb7eb', 'rootteststorages3queue_default', aliases=['azurite1', '3e8cfe9bf82e'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('3e8cfe9bf82ec94dba515c9e2f2f4c896518a702afa5c3c4a4efa5a5989fb7eb') (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue', service='azurite1', number=1) (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:Creating rootteststorages3queue_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 19:05:07 [ 396 ] INFO : Trying to connect to Azurite (cluster.py:3118, start) 2026-10-30 19:05:08 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': 'd335243a-d494-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:05:08 [ 396 ] DEBUG : Starting new HTTP connection (1): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:05:28 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': 'ded8b248-d494-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:05:28 [ 396 ] DEBUG : Starting new HTTP connection (2): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:05:52 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': 'ecf2b18a-d494-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:05:52 [ 396 ] DEBUG : Starting new HTTP connection (3): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:06:35 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '06dfa9a4-d495-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:06:35 [ 396 ] DEBUG : Starting new HTTP connection (4): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:06:35 [ 396 ] DEBUG : Can't connect to Azurite: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) (cluster.py:2703, wait_azurite_to_start) 2026-10-30 19:06:36 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '077a5f30-d495-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:06:36 [ 396 ] DEBUG : Starting new HTTP connection (1): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:06:54 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '11f0b946-d495-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:06:54 [ 396 ] DEBUG : Starting new HTTP connection (2): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:07:16 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '1f3ae752-d495-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:07:16 [ 396 ] DEBUG : Starting new HTTP connection (3): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:07:56 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '36dc116a-d495-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:07:56 [ 396 ] DEBUG : Starting new HTTP connection (4): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:07:56 [ 396 ] DEBUG : Can't connect to Azurite: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) (cluster.py:2703, wait_azurite_to_start) 2026-10-30 19:07:57 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '37774928-d495-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:07:57 [ 396 ] DEBUG : Starting new HTTP connection (1): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:08:17 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '43cade92-d495-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:08:17 [ 396 ] DEBUG : Starting new HTTP connection (2): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:08:41 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '51c24a44-d495-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:08:41 [ 396 ] DEBUG : Starting new HTTP connection (3): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:09:24 [ 396 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '6b9d9f40-d495-11f1-bd8e-1e66d2007ecd' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 19:09:24 [ 396 ] DEBUG : Starting new HTTP connection (4): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 19:09:24 [ 396 ] DEBUG : Can't connect to Azurite: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) (cluster.py:2703, wait_azurite_to_start) 2026-10-30 19:09:25 [ 396 ] DEBUG : Failed to start cluster: (cluster.py:3178, start) 2026-10-30 19:09:25 [ 396 ] DEBUG : Can't wait Azurite to start (cluster.py:3179, start) 2026-10-30 19:09:25 [ 396 ] DEBUG : None (cluster.py:3180, start) 2026-10-30 19:09:26 [ 396 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:113, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_azurite1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_minio1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_resolver_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy2_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo3_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo2_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_minio1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_resolver_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Stderr:Stopping rootteststorages3queue_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 19:09:48 [ 396 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml', 'down', '--volumes'] (cluster.py:113, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_azurite1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_minio1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_resolver_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy2_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo1_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo3_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo2_1 ... (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_resolver_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_minio1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing rootteststorages3queue_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing network rootteststorages3queue_default (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stderr:Removing volume rootteststorages3queue_data1-1 (cluster.py:123, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-10-30 19:09:50 [ 396 ] DEBUG : Docker networks for project rootteststorages3queue are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-10-30 19:09:50 [ 396 ] DEBUG : Docker containers for project rootteststorages3queue are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-10-30 19:09:50 [ 396 ] DEBUG : Docker volumes for project rootteststorages3queue are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-10-30 19:09:50 [ 396 ] DEBUG : Command:docker container list --all --filter name='^/rootteststorages3queue_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-10-30 19:09:50 [ 396 ] DEBUG : No running containers for project: rootteststorages3queue (cluster.py:904, cleanup) 2026-10-30 19:09:50 [ 396 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-10-30 19:09:50 [ 396 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-10-30 19:09:50 [ 396 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-10-30 19:09:50 [ 396 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-10-30 19:09:52 [ 396 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:113, run_and_check) 2026-10-30 19:09:53 [ 396 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 19:09:53 [ 396 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 19:09:53 [ 396 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 19:09:53 [ 396 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 19:09:53 [ 396 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 19:09:53 [ 396 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/.env', '--project-name', 'rootteststorages3queue', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_2/instance_24.5/docker-compose.yml', 'down', '--volumes'] (cluster.py:113, run_and_check) 2026-10-30 19:09:54 [ 396 ] DEBUG : Stderr:Removing network rootteststorages3queue_default (cluster.py:123, run_and_check) 2026-10-30 19:09:54 [ 396 ] DEBUG : Stderr:Network rootteststorages3queue_default not found. (cluster.py:123, run_and_check) 2026-10-30 19:09:54 [ 396 ] DEBUG : Stderr:Removing volume rootteststorages3queue_data1-1 (cluster.py:123, run_and_check) 2026-10-30 19:09:54 [ 396 ] DEBUG : Stderr:Volume rootteststorages3queue_data1-1 not found. (cluster.py:123, run_and_check) 2026-10-30 19:09:54 [ 396 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-10-30 19:09:54 [ 396 ] DEBUG : Docker networks for project rootteststorages3queue are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-10-30 19:09:54 [ 396 ] DEBUG : Docker containers for project rootteststorages3queue are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-10-30 19:09:54 [ 396 ] DEBUG : Docker volumes for project rootteststorages3queue are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-10-30 19:09:54 [ 396 ] DEBUG : Command:docker container list --all --filter name='^/rootteststorages3queue_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-10-30 19:09:54 [ 396 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-10-30 19:09:54 [ 396 ] DEBUG : No running containers for project: rootteststorages3queue (cluster.py:904, cleanup) 2026-10-30 19:09:54 [ 396 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-10-30 19:09:54 [ 396 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-10-30 19:09:54 [ 396 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-10-30 19:09:54 [ 396 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) _________ ERROR at setup of test_direct_select_multiple_files[ordered] _________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ________ ERROR at setup of test_direct_select_multiple_files[unordered] ________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ______________________ ERROR at setup of test_drop_table _______________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ________________ ERROR at setup of test_exception_during_insert ________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ___________ ERROR at setup of test_failed_retry[AzureQueue-ordered] ____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception __________ ERROR at setup of test_failed_retry[AzureQueue-unordered] ___________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_failed_retry[S3Queue-ordered] _____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ____________ ERROR at setup of test_failed_retry[S3Queue-unordered] ____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ______________________ ERROR at setup of test_max_set_age ______________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________________ ERROR at setup of test_max_set_size ______________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_multiple_tables_meta_mismatch _____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _______ ERROR at setup of test_multiple_tables_streaming_sync[unordered] _______ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception __ ERROR at setup of test_multiple_tables_streaming_sync_distributed[ordered] __ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _ ERROR at setup of test_multiple_tables_streaming_sync_distributed[unordered] _ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _______________ ERROR at setup of test_processed_file_setting[1] _______________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _______________ ERROR at setup of test_processed_file_setting[5] _______________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _________ ERROR at setup of test_processed_file_setting_distributed[1] _________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _________ ERROR at setup of test_processed_file_setting_distributed[5] _________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ______________ ERROR at setup of test_processing_threads[ordered] ______________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_processing_threads[unordered] _____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ___________________ ERROR at setup of test_s3_client_reused ____________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ____________________ ERROR at setup of test_settings_check _____________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ___________________ ERROR at setup of test_shards[ordered-1] ___________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ___________________ ERROR at setup of test_shards[ordered-8] ___________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception __________________ ERROR at setup of test_shards[unordered-1] __________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception __________________ ERROR at setup of test_shards[unordered-8] __________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_shards_distributed[ordered-1] _____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_shards_distributed[ordered-8] _____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ____________ ERROR at setup of test_shards_distributed[unordered-1] ____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ____________ ERROR at setup of test_shards_distributed[unordered-8] ____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ___________ ERROR at setup of test_streaming_to_many_views[ordered] ____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception __________ ERROR at setup of test_streaming_to_many_views[unordered] ___________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ______________ ERROR at setup of test_streaming_to_view_[ordered] ______________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_streaming_to_view_[unordered] _____________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ________________________ ERROR at setup of test_upgrade ________________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _______________________ ERROR at setup of test_upgrade_2 _______________________ @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ============================== slowest durations =============================== 307.13s setup test_storage_s3_queue/test.py::test_direct_select_file[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_upgrade_2 0.00s teardown test_storage_s3_queue/test.py::test_direct_select_file[unordered] 0.00s setup test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] 0.00s setup test_storage_s3_queue/test.py::test_shards_distributed[ordered-1] 0.00s teardown test_storage_s3_queue/test.py::test_settings_check 0.00s teardown test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_shards[ordered-1] 0.00s setup test_storage_s3_queue/test.py::test_upgrade_2 0.00s setup test_storage_s3_queue/test.py::test_shards_distributed[unordered-1] 0.00s setup test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] 0.00s setup test_storage_s3_queue/test.py::test_settings_check 0.00s setup test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_s3_client_reused 0.00s teardown test_storage_s3_queue/test.py::test_upgrade 0.00s setup test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] 0.00s setup test_storage_s3_queue/test.py::test_processing_threads[unordered] 0.00s setup test_storage_s3_queue/test.py::test_processing_threads[ordered] 0.00s setup test_storage_s3_queue/test.py::test_shards[unordered-1] 0.00s setup test_storage_s3_queue/test.py::test_shards_distributed[unordered-8] 0.00s setup test_storage_s3_queue/test.py::test_shards[ordered-8] 0.00s setup test_storage_s3_queue/test.py::test_shards[ordered-1] 0.00s setup test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] 0.00s setup test_storage_s3_queue/test.py::test_upgrade 0.00s setup test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_shards_distributed[unordered-1] 0.00s setup test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered] 0.00s setup test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_shards[unordered-8] 0.00s teardown test_storage_s3_queue/test.py::test_shards_distributed[ordered-1] 0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] 0.00s setup test_storage_s3_queue/test.py::test_s3_client_reused 0.00s teardown test_storage_s3_queue/test.py::test_shards[unordered-1] 0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] 0.00s setup test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered] 0.00s setup test_storage_s3_queue/test.py::test_shards[unordered-8] 0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_shards[ordered-8] 0.00s setup test_storage_s3_queue/test.py::test_drop_table 0.00s teardown test_storage_s3_queue/test.py::test_shards_distributed[unordered-8] 0.00s setup test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1] 0.00s setup test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] 0.00s setup test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] 0.00s setup test_storage_s3_queue/test.py::test_max_set_age 0.00s setup test_storage_s3_queue/test.py::test_max_set_size 0.00s teardown test_storage_s3_queue/test.py::test_processing_threads[ordered] 0.00s setup test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered] 0.00s setup test_storage_s3_queue/test.py::test_processed_file_setting[1] 0.00s setup test_storage_s3_queue/test.py::test_exception_during_insert 0.00s teardown test_storage_s3_queue/test.py::test_processed_file_setting[5] 0.00s setup test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 0.00s setup test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5] 0.00s teardown test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered] 0.00s setup test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5] 0.00s teardown test_storage_s3_queue/test.py::test_exception_during_insert 0.00s teardown test_storage_s3_queue/test.py::test_processing_threads[unordered] 0.00s setup test_storage_s3_queue/test.py::test_processed_file_setting[5] 0.00s teardown test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1] 0.00s teardown test_storage_s3_queue/test.py::test_max_set_age 0.00s setup test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered] 0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_drop_table 0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered] 0.00s teardown test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered] 0.00s teardown test_storage_s3_queue/test.py::test_max_set_size 0.00s teardown test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered] 0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_processed_file_setting[1] =========================== short test summary info ============================ ERROR test_storage_s3_queue/test.py::test_direct_select_file[unordered] - Exc... ERROR test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] ERROR test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] ERROR test_storage_s3_queue/test.py::test_drop_table - Exception: Can't wait ... ERROR test_storage_s3_queue/test.py::test_exception_during_insert - Exception... ERROR test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered] - ... ERROR test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered] ERROR test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered] - Exc... ERROR test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered] - E... ERROR test_storage_s3_queue/test.py::test_max_set_age - Exception: Can't wait... ERROR test_storage_s3_queue/test.py::test_max_set_size - Exception: Can't wai... ERROR test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch - Exc... ERROR test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] ERROR test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] ERROR test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] ERROR test_storage_s3_queue/test.py::test_processed_file_setting[1] - Excepti... ERROR test_storage_s3_queue/test.py::test_processed_file_setting[5] - Excepti... ERROR test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1] ERROR test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5] ERROR test_storage_s3_queue/test.py::test_processing_threads[ordered] - Excep... ERROR test_storage_s3_queue/test.py::test_processing_threads[unordered] - Exc... ERROR test_storage_s3_queue/test.py::test_s3_client_reused - Exception: Can't... ERROR test_storage_s3_queue/test.py::test_settings_check - Exception: Can't w... ERROR test_storage_s3_queue/test.py::test_shards[ordered-1] - Exception: Can'... ERROR test_storage_s3_queue/test.py::test_shards[ordered-8] - Exception: Can'... ERROR test_storage_s3_queue/test.py::test_shards[unordered-1] - Exception: Ca... ERROR test_storage_s3_queue/test.py::test_shards[unordered-8] - Exception: Ca... ERROR test_storage_s3_queue/test.py::test_shards_distributed[ordered-1] - Exc... ERROR test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] - Exc... ERROR test_storage_s3_queue/test.py::test_shards_distributed[unordered-1] - E... ERROR test_storage_s3_queue/test.py::test_shards_distributed[unordered-8] - E... ERROR test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] - ... ERROR test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] ERROR test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] - Excep... ERROR test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] - Exc... ERROR test_storage_s3_queue/test.py::test_upgrade - Exception: Can't wait Azu... ERROR test_storage_s3_queue/test.py::test_upgrade_2 - Exception: Can't wait A... ======================== 37 errors in 317.43s (0:05:17) ======================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 445, in subprocess.check_call(cmd, shell=True) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_shb54x --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS=" -rfEps --run-id=2 --color=no --durations=0 'test_storage_s3_queue/test.py::test_direct_select_file[unordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]' test_storage_s3_queue/test.py::test_drop_table test_storage_s3_queue/test.py::test_exception_during_insert 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered]' test_storage_s3_queue/test.py::test_max_set_age test_storage_s3_queue/test.py::test_max_set_size test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]' 'test_storage_s3_queue/test.py::test_processed_file_setting[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting[5]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5]' 'test_storage_s3_queue/test.py::test_processing_threads[ordered]' 'test_storage_s3_queue/test.py::test_processing_threads[unordered]' test_storage_s3_queue/test.py::test_s3_client_reused test_storage_s3_queue/test.py::test_settings_check 'test_storage_s3_queue/test.py::test_shards[ordered-1]' 'test_storage_s3_queue/test.py::test_shards[ordered-8]' 'test_storage_s3_queue/test.py::test_shards[unordered-1]' 'test_storage_s3_queue/test.py::test_shards[unordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-8]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]' test_storage_s3_queue/test.py::test_upgrade test_storage_s3_queue/test.py::test_upgrade_2 -vvv" altinityinfra/integration-tests-runner:cf95077f4609 ' returned non-zero exit status 1.